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HUMAN BEEVG PRESENCE DETECTION SYSTEM 
RELATIONSHIP TO OTHER APPLICATION 

This application claims the benefit of United States Provisional Patent Application Serial 
Number 60/454,296, filed March 13, 2003. 
BACKGROUND OF THE INVENTION 

Field of the Invention 

The invention relates generally to systems that detect the presence of a human being in a 
space of interest, and more particularly, to automatic detection of occupant status of a vehicle 
seat, i.e., whether the seat is occupied with a human being, or whether the seat is occupied with a 
non-human being (e.g., a box or a bag), or whether the seat is empty; determination when the 
seat is occupied by a human being whether the occupant is seated normally or leaning; 
determination of the presence of one or more human beings hidden or hiding within an enclosed 
space, such as a trailer, a cave, or an underground bunker; location of persons or animals trapped 
in a hazardous or hostile environment; and determination of the drowsiness of a human engaged 
in a dangerous task, such as the operation of an automobile or truck. 

Discussion of the Related Art 

In order that a front-seat passenger in a vehicle be protected in the event of a collision, 
motor vehicles are equipped with a front-seat passenger airbag. A necessary, but not entirely 
sufficient, condition for airbag deployment is that the front passenger seat be occupied. There 
are a number of systems for determining the presence of a human being in a vehicle seat. A 
typical detection system available in the art includes one or more sensing devices for measuring 
predetermined characteristics of a seat occupant. The sensed characteristics are used to 
determine whether a vehicle seat is occupied with a human subject, and further to determine 
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whether to deploy the airbag. The weight of the occupant has been used in known arrangements 
as a fundamental parameter in this regard. More particularly, weight is used as a criterion to 
distinguish between a human-occupied seat and an empty seat. However, weight-based occupant 
detection systems have met challenges in calibration for different seat types, different operating 
environments, etc. There also are known systems for detecting occupancy of vehicle seats in 
which vision, infrared, or ultrasonic focal plane array sensors are used to gather occupant 
information. These sensors are mounted within the vehicle, but away from the seats, such as in 
the overhead console. However, since these sensors do not directly measure physical attributes 
they are not as reliable as those that do, and accordingly, occupant-sensing systems based on 
such sensors are more prone to errors than their weight-based counterparts. Occupant-sensing 
systems that use focal plane array technology also tend to be much more costly. 

In another application of human being detection systems, unauthorized persons might 
endeavor to pass through entry checkpoints that have high security requirements (e.g., at airports, 
shipyards, border crossings, ballparks, concert halls, and secure or limited access areas such as 
military bases, power plants, nuclear plants, government buildings, etc) by hiding, for example, 
in the trunk of an automobile or in a trailer. Conventional visual security checks are time- 
consuming and prone to fault due to human error and fatigue. An automatic human being 
presence detection system can reduce the workload for the guards and provide information about 
the inside of an enclosed space. An enclosed-space human being detection system developed at 
Lockheed Martin Energy Systems, Inc. detects the presence of human beings with geophones 
placed on the vehicle and employs wavelet analysis to the sensed data to determine if there are 
persons in the vehicle. This known system, as is the case with others, is capable of detecting 
human being presence, but it needs careful positioning and tuning of the sensors. 
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Enclosed spaces not only include boxed areas, but also areas that are hemispherical, 
tubular, etc. Examples of the other kind include caves, underground bunkers, turmels, etc, A 
case in point is the ongoing anti-terrorism search for hidden/hiding criminals over the 
countryside and mountain ranges of certain foreign countries. The system proposed herein is 
useful in locating and capturing human beings who are hidden or hiding in caves, underground 
bunkers, tunnels, etc. 

There is a need for a human being presence detection system that locates human beings 
buried under rubbles, trapped behind barriers, or inside buildings. After a disaster strikes, such 
as an earthquake, a hurricane, or a terrorist attack, human beings might be buried or trapped 
under rubble or behind large barriers. Similarly, there is a need to find human beings trapped 
indoors during a building fire. Rapid location of such persons can reduce the loss of life. 

hi a "life-detection system," human subjects are illuminated by penetrating microwave, 
and the reflected wave is modulated by the body movements, including the breathing and 
heartbeat. In this arrangement, the human being presence detection is accomplished by 
extracting the breathing and heartbeat component signals from the received microwave signal. 

Drowsiness is a common attribute of humans engaged in repetitive monotonous tasks. 
When the task is, for example, the operation of an automobile or truck, the consequences of 
undetected drowsiness can be fatal. Previous approaches to the detection of drowsiness have 
relied on measuring eye closure, /.e, so-called "perclose." Although perclose can be a reliable 
measure of human drowsiness, it cannot be reliably estimated in operating conditions and 
requires very careful calibration for each human. 

Heartbeat and breathing signals have been extensively used in detecting and monitoring 
human beings. In one known arrangement the author proposed a hand-held acoustic sensor pad 
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that is placed on the subjects' upper chest to monitor heartbeat and breathing patterns. Since the 
water-filled sensor is excellently coupled with the human body, it is able to collect high signal- 
to-noise ratio heartbeat and breath signals. In another known arrangement, the life detection 
system utilizes active microwave sensors to acquire the. heartbeat and breathing signals with a 
high signal-to-noise ratio. In both these cases, the acquired signal clearly shows the breathing 
and heartbeat patterns in time domain. 

SUMMARY OF THE INVENTION 

To detect the presence of human subjects without intruding on their privacy or without 
being able to access them directly, the sensors of the detection system should be able to take 
advantage by remote operation of some reliable featured movement of the human body or its 
parts. The heartbeat signal inside the human body can propagate to the body surface and then 
generate a shock wave around the body. This wave can be used as evidence of the presence of a 
human being. Each time the heart beats it generates a small measurable shock wave, 
ballistocardiogram, that propagates through the body. This wave can be measured either with a 
sensor that is directly in contact with the human body or with a sensor that is not in direct contact 
with the body but which has a receiver that is appropriately positioned and oriented. Human 
breathing is similarly detectable. Pressure transducers are the sensors of choice for measuring 
such human heartbeat and breathing signals. Such pressure transducers include, for example, 
strain gauges, load cells, accelerometers, geophones, laser vibrometers, fiber-optic probes, 
microwave radiometers, etc. 

In the applications of the present invention, the heartbeat and breathing signals are very weak 
compared to the ambient noise and therefore cannot be detected by examining the time domain 
signal alone. To overcome the extremely low signal-to-noise ratio ubiquitous to the application 
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under discussion, a novel non-linear short-term frequency-selective energy distribution method is 
used to detect the presence of heartbeat and breathing signals embedded in the measured data. 
This novel method has the ability to detect weak heartbeat and breathing signals in a variety of 
different noises, e.g., wind, suspension rocking, road, etc. 

The human being presence detection system accepts data from pressure transducers 
located in an area of interest. A novel short-term non-linear frequency-selective energy 
distribution method is applied to detect the existence of heartbeat and breathing signals in the 
measured data. This method, in conjunction with measurement transducers, forms a system for 
detecting the presence and absence of human beings in any reasonably enclosed space. 

In one embodiment of the invention, the human being presence detection system 
distinguishes between human being-occupied and empty vehicle seats, and as such can be 
integrated into a driver/passenger restraint (seat-belt) and protection (airbag) system. 

In another illustrative embodiment of the invention, the human being presence detection 
system determines the presence of human subjects in an enclosed space, such as a trailer, a 
vehicle trunk, etc. This detection capability is useful for entry point security screening purposes. 

In yet another illustrative embodiment of the invention, the human being presence 
detection system locates and facilitates the rescue of human beings trapped under rubble, behind 
barriers, inside buildings, etc. With appropriately chosen and positioned pressure transducers, 
the human heartbeat and breathing signal will be recorded, and with the non-linear short-term 
frequency-selective energy distribution method disclosed herein, the presence of human beings is 
detected. 

In yet another exemplary embodiment of the invention, the human presence detection 
system can be used to detect human drowsiness/fatigue. This detection capability can be used to 
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prevent accidents and loss of life when the human is engaged in dangerous activity such as 
driving an automobile or truck. 

In a further illustrative embodiment of the invention, the human being presence detection 
system identifies whether a human vehicle seat occupant is properly seated - in-position 
(appropriately away from the airbag) or out-of-position (too close to the airbag). 

Some of the distinguishing features of the human being presence detection system in 
accordance with the invention disclosed herein are its: 

1. use of existing pressure transducer data in detecting human being presence in a vehicle 
seat; 

2. distinguishing between a human being-occupied and an empty vehicle seat without 
requiring extemally mounted focal plane array sensors, such as vision, infrared, 
ultrasonic sensors, etc,\ 

3. avoidance of the need for calibration and the ambiguity associated with weight-based seat 
occupancy detection systems; 

4. real time inspection of an enclosed space; 

5. utility in locating and facilitation of the rescue of trapped human beings in spaces that are 
difficult for human access such as debris/rubble, large barriers, buming buildings, etc,\ 

6. utility in locating and facilitation of the capture of human beings hidden/hiding in 
enclosed spaces such as caves, underground bunkers, tunnels, etc,\ 

7. ability to detect drowsiness and fatigue in humans engaged in monotonous or tedious 
tasks, and 
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8. robust non-linear short-term frequency-selective energy distribution analysis and 

appearance-recognition methodologies for weak signal, large noise, and varying 

operating conditions. 
BRIEF DESCRIPTION OF THE DRAWINGS 

Comprehension of the invention is facilitated by reading the following detailed 
description in conjunction with the annexed drawings, in which: 

Figs, la and lb are graphical representations that illustrate the energy distribution across 
various frequencies of human-being-present and empty signals in a vehicle seat occupancy 
detection application; 

Figs. 2a, 2b, 2c, and 2d are graphical representations that illustrate the signal data 
obtained from four pressure transducers (sensors 1-4, respectively), illustratively in the form of 
load cells, their corresponding short-time frequency-selective energy distribution, and an 
individual human being presence/absence decision; 

Figs. 3a and 3b are graphical representations that illustrate the composite decision 
obtained by fusing all four individual decisions; 

Figs. 4a, 4b, 4c, and 4d are graphical representations that illustrate a general result of 
human being presence detection in accordance with the invention; and 

Fig. 5 is a simplified schematic illustration of a specific illustrative arrangement of 
certain structural elements arranged in accordance with the principles of the invention. 
DETAILED DESCRIPTION 

1. Data Acquisition 

Each time the heart beats the ballistocardiogram propagates through the human body, 
through the media between the human body and the measurement transducer, and is eventually 
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recorded by the transducer. The same is true of the human breathing signal. The media between 
the human body and the measurement transducer vary tremendously across the different 
appHcations Such media often includes the seat cover, cushion, and seat frame combination in 
vehicle seat occupancy detection applications; the vehicle body that encloses the trailer or the 
trunk space in entry point inspection applications; the rubble surface or the glass pane window in 
search and rescue applications; or the earth or the outer wall in search and capture applications. 
Since ballistocardiogram and breathing signals propagate in these media with different 
attenuations, the selection of the transducers that measure these signals is application 
dependent. For example, the vehicle seat occupancy application uses a load cell, 
accelerometers, strain gauges, etc\ the entry point security screening application uses geophones, 
acoustic pads, etc.\ the locate and rescue or capture applications use geophones, fiber-optic 
probes, laser vibrometers, microwave radiometers etc. No matter which transducer is used, they 
all produce at their respective outputs a real-valued signal sequence data over time. The data is 
subsequently processed by the human being presence detection system. 

2. Human Being Presence Detection Algorithm 

The nature of the data recorded will vary considerably across the gamut of pressure 
transducers. In particular, the strength of the heartbeat and breathing signals will depend on the 
location/proximity of the human being, the media between the human being and the transducer, 
the sensitivity of the transducer, and its frequency response. The environments in which these 
transducers operate could introduce a variety of noise signals of varying strengths and energy 
distributions. However, all such measured data do provide information about the presence of 
human beings, and the redundant and complementary nature of these transducers allows us to 
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improve the accuracy and reliability of human being presence detection with data fusion 
techniques. 

There are several categories of data fusion methods based on the stage at which the 
fusion is performed, namely, signal-based, feature-based and decision-based. Since the sensors 
in the present detection setting are conditionally independent in their sensing capability, 
decision-based fusion is the method of choice. In decision-based fusion, each sensor makes a 
separate detection decision. These decisions are then combined using voting techniques, as 
described hereinbelow: 

Let xX[n]t^^) denote the signal obtained with the i^^ sensor at sampling time instant 

«/^^ where z = l,2,---,/, and / is the total number of sensors. At any sampling instant no, a 

decision regarding the presence/absence of human beings will be made based on the previous Ni 
signal samples received: 

Where A^, is determined by the sampling rate of the i^^ signal, with the assumption that the 
energy distribution of Ni continuous samples of the measured signal x.([n]t^^) will provide 
enough information about the existence of heartbeat and breathing in 3c,- (wj . 

The Ni -point non-linear discrete Fourier transform of the signal 3c,. («o) is given by: 

X:^{k) = -= tf[xXn)]e ,k = W,-,N,-\ 

Where ffj denotes the mean-subtraction operation, followed by a non-linear operation such as 
the absolute value of the mean subtracted signal. Figure la illustrates the energy distribution 
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across various frequencies when there's human present versus empty. It is obvious that the signal 
containing a heartbeat concentrates its energy in select frequencies in the 7-2 Hz frequency 
range. In this application, it turns out that noise signals - wind, suspension rocking, road, etc. - 
result in a signal components of very low frequency, mostly under 1 Hz or over 2 Hz. To be 
immune of such low frequency noise, the human presence detection algorithm will focus on the 
frequency range from 0 to 4 Hz, For comparison. Figure lb illustrates the energy distribution of 
the original signal over the same band of frequencies. The effectiveness of the non-linear 
operation is clear. 

Let r denote the select frequencies over which the human heartbeat and breathing signals 
have sufficient/discernible energies. The non-linear normalized energy distribution of 

3c. over these select frequencies of T is given by Xn^ = {|| X'^^{k) \\^ ,k e T} . If F'ls selected 

carefully, the short-term frequency-selective energy of the measured signal is a good 
discriminant between the presence and absence of human heartbeat and breathing signals in the 
measurement. 

As an example, let 

e,(v.)=HElKWir. 

We call e/(«o^,.) non-linear short-term frequency-selective energy of the measured signal at 
time instant n^t^^ .An appropriate threshold Ei can be used to determine the presence of a human, 
i.e., if e^{nQt^^)> E., then c?,(«o^,) = 1> which means that the presence of human is detected; 
otherwise, d^in^t^^ ) = 0 , which means that no human is detected in the transducer measurement. 
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When the above individual sensor detection algorithm is applied to measurement data 
obtained from all the fielded sensors, at any instant in time / independent decisions are available. 
To make the most of all the sensor measurements, a decision-based data fusion approach is 
employed. A voting technique is employed to integrate the individual decisions. A variety of 
voting techniques are available: 

At least one ^ t/. {n^t^^ ) > 1 , 

1=1 

Majority Y.d,{n,t„)> 1 12, 

/ 

All ^c/,(V.,) = /,e^c. 

1-1 

3. Human Being Presence Detection Results 

The human being presence detection algorithm described above has been applied to the 
vehicle seat occupancy detection application. Fig. 2 shows the signal data from the four pressure 
transducers (load cells), their corresponding short-time frequency-selective energy distribution, 
and the individual human being presence/absence decision. The composite decision obtained by 
fusing all four individual decisions is shown in Fig. 3. As a comparison, the weight of the 
occupant calculated from the load sensors is provided. It can be seen that the detection result 
matches well with the weight calculation. Fig. 4 provides a more general result of human being 
presence detection using the invention disclosure herein. A single pressure transducer is used to 
record measurements of the human heartbeat and breathing signal when the human being enters 
and exits an area of interest and also when a human being-like dummy is placed instead of the 
human being. Notice that the time domain signal offers virtually no discrimination between the 
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various situations, whereas the short-time frequency-selective energy offers an excellent 
discrimination. 

4. Occupant Pose Classification Algorithm 

Once a human being is determined to be present in the enclosed space of interest, other 
characteristic of the human being are also of interest. For example, in the vehicle seat occupancy 
application, once it has been determined that the seat is occupied by a human being, whether the 
occupant is normally seated or leaning is also of interest. It is envisioned that if the occupant is 
normally seated then the airbag will be deployed with full force, whereas if the occupant is 
leaning the airbag will be deployed with lower power or not at all. 

Categorization of the detected human being is also done using the short-time measured 
signal 3c. (Wq) . Details follow: 

4.1 Database, Dimensionality Reduction, and Signal Representation 

— — 1 — 2 — / I 

Let yin^) = y( n^,X n^,..,,X J. For simplicity and clarity of further description, we will 

not specify time no where no ambiguity rises, Le,, y ^ , . A training set ^consists 

of samples of y for the various human being categories. Let mj, mj, ... , represent the 

number of samples of y in each of the c human being categories, T^Tj ...7^ denote the partition 

of the training samples into c categories (i.e., T = r, ...uT^), and M denote the total 

number of samples in T. Let t represent the corresponding human being category for each 
training sample y . 

Using a Fisherbasis algorithm, we compute a projection matrix P to project the training 
samples T onto a much lower dimensional space 9^^ Computation of the projection matrix 
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f 

involves /7, = , / = Ur-c, = Yi^^Hy.erp^ -Mi)()^k "J^i) . and 

^5 = Zili'^'fe ^yavgjpi -yavgj . where y^^ = ^^^^yj/M is the average signal vector. The 
projection matrix P is chosen so that {PS^P^y\PS^P^) is maximized. This amounts to 
computing the eigenvectors of the matrix S^^S^y which is a large NxN matrix (where 
denotes the length of y ) whose eigenvectors are not easily determined. An alternate strategy for 
computing of P is pursued. 

Let Z = [J/. -y^^gJ = ly2,--M] be an NxM dimensional matrix of zero mean short- 
time measurements. The covariance matrix of Z is then given byC = ZZ^,aA^x7V matrix. The 
alternate strategy for computing P involves finding the eigenvectors of C. In reality, since 
M «N, the eigenvectors of the MxM matrix Z^Z is found. The non-zero eigenvectors 
^1,62,...,^^ of the covariance matrix C are then computed as Z *eig[z^z). The vectors 
e^,e2y,,.,e^ are typically called eigenbases, which are unit norm and sorted in the order of 
decreasing eigenvalues. If P^ = [^x^^i^-'^^m-i] matrix of eigenbases corresponding to the 
M-2 largest eigenvalues, then the matrices Sw ^P^S^P^ and Sb =PeSbPe effective 
measures of and Sg in the eigenspace. Finally, the Fisher projection matrix P is computed 
as P = eig(Sw''SB\ 

Once the projection matrix P is computed, each of the training signals yj can be 
projected onto a lower-dimensional vector as follows: Pj = P^pj . 
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4.2 Classification 

Given the non-linear energy spectrum measurement signal y , with human presence, it is 
first projected onto a lower-dimensional space using P - p^P^y . The human category of y is 
decided by a nearest-neighbor classification of the low-dimensional vector p . 

Figs, la and lb are graphical representations that illustrate the energy distribution across 
various frequencies of human-being-present and empty signals in a vehicle seat occupancy 
detection application. Fig. la illustrates the energy distribution using the preferred nonlinear 
energy distribution methodology of the present invention, and Fig. lb illustrates the original 
energy distribution. It is evident that the distinction between an empty environment (dotted 
graph) and the same environment occupied by a human (continuous line graph) is significantly 
more evident in the nonlinear energy distribution analysis. 

Figs. 2a, 2b, 2c, and 2d are graphical representations that illustrate the signal data 
obtained from four pressure transducers (sensors 1-4, respectively), illustratively in the form of 
load cells, their corresponding short-time frequency-selective energy distribution, and an 
individual human being presence/absence decision. 

Figs. 3a and 3b are graphical representations that illustrate the composite decision 
obtained by fusing all four individual decisions determined in Figs. 2a, 2b, 2c, and 2d. As 
shown, using the nonlinear energy distribution analysis of the present invention results in a 
consistent unequivocal determination of presence decision making using all four transducers. 

Figs. 4a, 4b, 4c, and 4d are graphical representations that illustrate a general result of 
human being presence detection in accordance with the invention. Fig. 4d is useful to illustrate 
that the human presence detection of Fig. 4c corresponds to the ground truth represented in Fig. 
4d. 
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Fig. 5 is a simplified schematic illustration of a specific illustrative arrangement of 
certain structural elements arranged in accordance with the principles of the invention. As 
shown in this figure, a predetermined space in the form of a vehicle interior 50, within the 
dashed line in the figure, has a vehicle seat 52 therein, as well as a conventional steering wheel 
54. There are provided on a seat frame 56 a pair of pressure transducers 60 and 62 that produce 
respective transducer signals responsive to the pressure applied to vehicle seat 52. The 
transducer signals are propagated to a processor 65 that computes one or more algorithms, as 
hereinabove described. 

In accordance with the invention, there are additionally provided further transducers 67 
and 68 within vehicle interior 50. An exterior transducer 69 is also shown in the figure to be 
provided. Transducers 67, 68, and 69 produce signals responsive to the location and 
movement of a person (not shown) on the vehicle seat, all of which signals being propagated 
to processor 65. 

As stated, processor 65 employs data from a database memory 75 to compute, in some 
embodiments, an indication of the presence or absence of a human being on the vehicle seat. 
In other embodiments, the data from database memory 75 is used to identify certain 
behavioral characteristics of the human being, such as the level of alertness. Generally, the 
processor will make a determination between binary conditions, such as present or not 
present, or alert or not alert. Thus, the database memory will generally contain at least two 
memory regions 77 and 79 that contain data that would correspond to the two conditions 
being determined by the processor. For example, in an embodiment of the invention where 
the presence of a human being on the vehicle seat is to be determined, the first data would 
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correspond to data representative of the human present condition, and the second data would 

correspond to data representative of the human absent condition. 

Although the invention has been described in terms of specific embodiments and 
applications, persons skilled in the art may, in light of this teaching, generate additional 
embodiments without exceeding the scope or departing from the spirit of the claimed invention. 
Accordingly, it is to be understood that the drawing and description in this disclosure are 
proffered to facilitate comprehension of the invention, and should not be construed to limit the 
scope thereof 
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